The present invention relates to digital color image enhancement and in particular restoration and recovery of lost contrast and details in highlights due to overexposure or clipping in images and footage recorded digitally with smart phones equipped with a camera, dedicated digital cameras (DLSR, compact cameras) or digital video cameras and thereby extending the usable dynamic range.
Loss of detail and contrast in highlights is a well-known and undesirable phenomenon among practitioners such as photographers and videographers.
Due to inherent limitations of the recording device, such as limited dynamic range, device design choices made in respect to exposure evaluation, and processing methodology, video footage, such as encoded MP4, H264, and AVCHD, or still images such as JPEG and TIFF, often exhibit unintentional loss of detail and contrast in highlights.
The present method rely on the novel observation that pixels in an overexposed area of an image often contain some valid signal residual in at least one color component and thereby offer an opportunity to restore said overexposed areas of an image with a visually pleasing result. The method is believed to be novel in respect to its application to highly processed and compressed low-bit data that is characteristically available with file formats such as JPEG and video files such as H264, AVCHD encoded files.
The invention is used with computer software used for digital video editing or editing of digital still photographs providing a user interface and display to validate the result of parameter changes. The invention can also be used and implemented as a processing step in an image acquisition device without any user intervention.
Various ad-hoc techniques are well known to alleviate loss of highlight details:    1. Premeditated under-exposure during acquisition. The exposure is then gained in a controlled fashion in a post-processing environment in order to gain back the overall and intentional brightness of the scene without causing overexposure or clipping. The main disadvantage of this approach is that the resulting image will usually exhibit more noise throughout the tonal range especially in shadow areas.    2. Use of a photographic gradation filter in from of the lens so that highlight detail in, for instance, the sky is preserved.
A common approach, especially for digital cameras, is to offer a RAW setting that will record images in a so-called RAW file. Such RAW files contain minimally processed data from the imaging sensor. RAW files are so named because they are not yet processed and therefore are not ready to be printed or displayed. RAW images are processed by a RAW converter where precise adjustments, such as exposure, highlight recovery can be performed before conversion to a regular image file format such as JPEG for storage, printing, or further manipulation. The main disadvantage of such RAW images is that they are not immediately available for display or sharing, such as uploading to social media services or web galleries (i.e. Facebook, Flicker) without first having to undergo RAW conversion. It is well known in the art of RAW conversion to restore highlight detail and contrast based on the minimally processed data available in the RAW file. Typically RAW data will is high-bit (10-16 bit) and have not undergone any color processing thereby making the effort of restoring highlight information a much more attainable and predictable task.
Another common approach is to use so-called HDR techniques. HDR is a range of methods to provide higher dynamic range from the imaging process. Non-HDR cameras acquire images at one exposure level with a limited contrast range. This results in the loss of detail in highlights or shadow areas of a scene, depending on whether the camera had a low or high exposure setting. HDR compensates for this loss of detail by acquiring multiple images at different exposure levels and intelligently blend them together to produce an image that covers the full tonal range without loss of details or contrast. The main disadvantage of using HDR techniques is that they are limited in scope to mainly static subject matter. For instance, if the subject is moving in-between two consecutive exposures it will cause an undesirable “ghosting” effect when such two images are blended together.